Leveraging a rolling cross-section design in panel studies to investigate police legitimacy

Monica Gerber and Cristóbal Moya



London School of Economics and Political Science

May 22, 2023

Today

  1. Motivation
  2. ‘Unexpected event during survey’ design
  3. The rolling cross-section design
  4. This Study: a rolling cross-section panel design
  5. Implementation challenges

Motivation

Confirmatory research

  • Inferences about changes in police legitimacy
  • Panel study
    • Interview individuals at two or more points in time

Exploratory research

  • Unexpected events relevant to police legitimacy
  • Rolling cross-sectional design (RCS)
    • Randomized controlled interviewing over time

‘Unexpected event during survey’ design

“research design that exploits the occurrence of an unexpected event during the fieldwork of a public opinion survey to estimate its causal effect on a relevant outcome by comparing responses of the individuals interviewed before the event \(t_i<t_e\) (control group) to those of respondents interviewed after the event \(t_i>t_e\) (treatment group).” (Muñoz et al., 2020, p. 187)

  • Quasi-experiment: Effects of naturally occurring events that cannot be manipulated through controlled experiments
  • Assumption that interview time is independent from the time when the event occurs

The RCS design

The “expected-unexpected” event during survey design

“the design is just a standard cross-section, but the day on which a respondent is interviewed is chosen randomly” (Johnston & Brady, 2022, p. 283)

“The data collected in an RCS allow a fine-grained analysis of dynamic phenomena of public opinion” (Lutz et al., 2013, p. 3)

  • Temporal heterogeneity as an opportunity to move closer to causal inference (Johnston & Brady, 2002)

The RCS design

The “expected-unexpected” event during survey design

Steps (Kenski et al., 2010):

  1. Total number of interviews is established
  2. Full sample is randomly divided into multiple replicates
  3. Replicates assigned to be interviewed at different intervals of times
  4. The same protocol is followed for all replicates

The rolling cross-section design

Example

The RCS design

Assumptions

  1. Temporal ignorability: Moment of the interview should be as good as random

  2. Excludability: Timing of the interview \(t\) affects \(Y\) only through event \(T\)

  3. Compliance: assignment to the treatment (\(t_i>t_e\)) corresponds to actually receiving treatment (\(T=1\))

The RCS design

Assumptions

  1. Temporal ignorability: “for any individual, the potential outcomes must be independent from the moment of the interview” (Muñoz et al., p. 189)
  • Moment of the interview should be as good as random
  • In RCS moment of interview is random
  • Protocol to ensure equal procedures to contact and recontact sample units
  • Need to compare ‘treated’ and ‘not-treated’ in terms of ease of contact and socio-demographics (Lutz et al., 2013)

The RCS design

Assumptions

  1. Excludability: “any difference between respondents interviewed before and after the event shall be the sole consequence of the event” (Muñoz et al., p. 189)
  • Timing of the interview \(t\) affects \(Y\) only through event \(T\)
  • Beware of collateral events, simultaneous events and unrelated time trends
  • Need to characterize the event and its reactions

The RCS design

Assumptions

  1. Compliance: “Assignment to the treatment group \(t_i>t_e\) was assumed to perfectly correspond to actually receiving treatment \(T=1\).” (Muñoz et al., 2020, p. 194)
  • Need to measure compliance

This Study

Police legitimacy

  • Police work requires public support to be able to function in a democratic context
  • Beliefs that the police act in procedurally just ways and are legitimate facilitate compliance and cooperation with authorities
  • A socially legitimate police force should require little violence to confront opposition
  • However, the world has seen many cases of violent demonstrations, police violence and human rights violations

This Study

The Chilean case

  • Chile’s main police force, the Carabineros, was among the most trusted institutions (Dammert, 2019)
  • Yet, recent events of corruption and excessive use of force during the social uprising in October 2019 have produced a serious loss of trust (Gerber et al., 2023)
  • Crisis of legitimacy: proposals of “deep reform” and even the dissolution of the institution
  • Need to understand relationship between police and citizens in Chile during the next crucial years

This Study

Aims

  • Confirmatory:
    • Track over time people’s perceived police legitimacy and motivational postures towards the police, and examine their relationships with perceived procedural justice and fear of the police in Chile
  • Exploratory:
    • Determine the effects of potential unexpected events (such as police violence, corruption scandals or reforms) on perceptions of legitimacy, procedural justice and fear of the police in Chile.

This Study

A RCS panel design

  • Probability-based online panel of the population in Chile (18-65 years old)
  • Panel study with 10 waves
  • Face-to-face recruitment with probability-based sampling design
  • Sample size
    • Expected sample in contact survey: n = 5,300
    • Expected sample in Wave 1: n = 1,500
  • Possibility to combine with non-probabilistic sample to increase sample size

This Study

A RCS panel design


This Study

A RCS panel design


This Study

A RCS panel design


This Study

Identify unexpected event with RCS design

This Study

Identify unexpected event with panel design

This Study

Identify unexpected events with RCS panel design

This Study

Implementation challenges


This Study

Implementation challenges


Thank you!

Contact: